Generalizability of Causal and Statistical Relations

The problem of generalizability of empirical findings (experimental and observational) to new environments, settings, and populations is one of the central problems in causal inference. Experiments in the sciences are invariably conducted with the intent of being used elsewhere (e.g., outside the laboratory), where conditions are likely to be different. This practice is based on the premise that, due to certain commonalities between the source and target environments, causal claims would be valid even where experiments have never been performed. Despite the extensive amount of empirical work relying on this premise, practically no formal treatments have been able to determine the conditions under which generalizations are valid, in some formal sense. Our work develops a theoretical framework for understanding, representing, and algorithmizing the generalization problem as encountered in many practical settings in data-intensive fields. Our framework puts many apparently disparate generalization problems under the same theoretical umbrella. In this talk, I will start with a brief review of the basic concepts, principles, and mathematical tools necessary for reasoning about causal and counterfactual relations [1, 2, 3]. I will then introduce two special problems under the generalization umbrella. First, I will discuss “transportability” [4, 5, 6], that is, how information acquired by experiments in one setting can be reused to answer queries in another, possibly different setting where only limited information can be collected. This question embraces several sub-problems treated informally in the literature under rubrics such as “external validity” [7, 8], “meta-analysis” [9], “heterogeneity” [10], “quasi-experiments” [11, Ch. 3]. Further, I will discuss selection bias [12, 13, 14], that is, how knowledge from a sampled subpopulation can be generalized to the entire population when sampling selection is not random, but determined by variables in the analysis, which means units are preferentially excluded from the sample. In both problems, we provide complete conditions and algorithms to support the inductive step required in the corresponding task. This characterization distinguishes between estimable and non-estimable queries, and identifies which pieces of scientific knowledge need to be collected in each study to construct a bias-free estimate of the target query. The problems discussed in this work have applications in several empirical sciences such as Bioinformatics, Medicine, Economics, Social Sciences as well as in data-driven fields such as Machine Learning, Artificial Intelligence and Statistics.